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ABSTRACT 



The evaluation of instruction via distance learning at the 
high school and college level offers special challenges, particularly the 
evaluation of one-way video- -two-way audio instructional programs. The role 
of the teacher is radically changed and teaching strategies and course design 
are particularly important in a distance leaning environment. Students must 
be prepared to assume more responsibility for their own learning since, 
usually, there is no opportunity for immediate clarification as in a normal 
classroom setting. The beginning of the class is particularly important in 
setting the stage for learning. Teachers also need to plan for as much 
student interaction as possible. Effective teaching includes intentional use 
of modeling behaviors and illustrations or demonstrations. Teachers must also 
make opportunities for checking student understanding at the most opportune 
time often accomplished via telephone or computer. In television instruction, 
the effective use of time must be evaluated differently, with more extended 
wait times when questioning distance students. Communication in a distance 
format minimizes the nonverbal component and raises the potential for 
erroneous interpretation. Independent practice to extend content is 
particularly important in the distance learning environment but is more 
difficult to incorporate into the lesson. Well-researched criteria are needed 
to evaluate the effectiveness of distance learning instruction. (Contains 38 
references.) (DB) 
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Observation of Instruction via Distance Learning: The Need for a New 

Evaluation Paradigm 

Paradigms used in the observation and evaluation of regular classroom teaching may not 
be applicable when observing instruction via distance learning. The body of knowledge gleaned 
from the research on effective teaching when applied to teaching on television may simply not fit. 
At the very least, teacher behaviors tried and tested in a regular classroom will need modification 
if they are to be successful in distance learning. This is especially true when the teaching is to be 
performed in a one-way video mode. In this mode, the student must assume more of the 
responsibility for learning. Teaching on television is usually conducted in a one-way video— two- 
way audio, or two-way interactive video format. This article deals with the key issues that are 
essential to quality instruction via distance learning. As teaching and learning address the urgency 
of the technological explosion, compatible accommodations must be made for impending changes 
Instruction via distance learning needs close scrutiny and critical evaluation as it holds the 
potential to change teaching and learning. Although the costs may be prohibitive, the impact for 
teachers and learners is far too great to ignore. It is not a panacea; neither should it be dismissed 
with an attitude of “this too shall pass!” 

Distance education is a distinguished system of education, distinguished from other 
educational forms by its separation in both time and place and by its teaching and learning acts 
(Rumble, 1989). The teacher is a very important element in the educational process, whether in a 
traditional classroom or in a distance education setting. One of the ways to determine the 
effectiveness of instruction conducted on a distance learning system might be to observe and 
document the characteristics and behaviors of distance learning instructors for comparison to the 
teaching behaviors found in the research on effective teaching. Specific teaching behaviors such 




3 



2 



as providing effective praise (Brophy, 1981), use of advance organizers (Ausubel, 1960) or 
frequency of reviews (Cruickshank, 1986) might serve as starting places. Questions about 
effective teaching might include inquiry into interaction such as: 1) how often does the instructor 
initiate interaction with each student, 2) how often do the students initiate interaction with the 
instructor, 3) how much wait time does the instructor give a student responding to a question, 4) 
how often does the instructor use advance organizers, or 5) how much time does the instructor 
spend in reviewing previous lessons? Questions like these might be more useful in an observation 
instrument for teaching on television than an instrument normally used in a regular classroom. An 
analysis of the amount of teacher dialogue compared to the amount of student dialogue or a 
determination of how much of the interaction the teacher initiated and how much the students 
initiated might also prove beneficial (Barker, 1988). 

Several studies assessing student attitudes toward media and technology in the distance 
learning environment have been conducted (Allen, 1995, Bangpibob, 1995, and Bozik, 1995). 
Adult attitudes toward instructional technologies are positive (Dillon, Haynes & Price, 1990); 
however, other studies indicate student attitudes are influenced by their familiarity with the 
technologies employed (Riddle, 1990, Smith & McNeils, 1993). Teachers and learners must 
realize computers are not inherently interactive. They provide an excellent environment for 
discussion, but they are only interactive if students participate responsively and regularly 
(Eastmond, 1995). 

Kinzie, Delcourt and Powers (1994) found attitudes are important predictors of success 
and are critical areas for future examination. Distance learning students express a “need to know 
classmates” and a desire to “interact with someone else in class” (Egan & Sebastian, 1993). The 
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practice most often mentioned by students in a description of outstanding electronic teaching 
practices was the practice of providing for student-to-student and instructor-to-student interaction 
(Western Cooperative for Educational Telecommunication, 1995). Clearly, an examination of 
instructional effectiveness must consider an assessment of student attitudes toward the learning 
delivery system. 

Teaching strategies and course design influence student attitudes in a distance learning 
environment. Students are positive toward interactive teaching methods (Burge, 1994); however, 
their attitudes toward the impact of technological difficulties are negative (Riddle, 1990). Saba 
and Shearer (1994) found students ‘Telt connected and satisfied in a computer-facilitated 
interactive video classroom, yet felt isolated from the main class and saw themselves as passive 
observers in a video-only classroom.” Keegan’s comment might serve as a summation: 

“Interactive learning environments have proven difficult to design and deliver; however, current 
instructional emphases must consider new interactive technologies if “transactional distance” 
between learners and instructors is to be bridged ” (Keegan, 1993). 

The one-way video— two-way audio format necessitates conveying of information where 
the nonverbal dimension of teaching is missing for the most part (the student can see the teacher, 
but the teacher cannot see the student). The role of the teacher is radically altered. The teacher 
must make illustrations by describing or by creating verbal pictures for the students. Pinney’s 
research (1969) found that without the opportunity to see the student, the teacher must prepare 
the students in such a way that they assume more responsibility for their own learning. Usually, 
there is no opportunity for immediate clarification as one would do in a normal classroom Setting. 
This represents a major adjustment if the teacher uses a research based teaching approach. 
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Preparing the students to learn is an essential element in effective teaching. The beginning 
of the class is an important time. First impressions tend to have lasting effects, and teaching at the 
beginning of the period might be much more influential than the instruction taking place later. The 
beginning of class is one of the most critical “teachable moments.” The teacher must take this into 
account and work diligently to get the students ready to learn (Hunter, 1982). 

Whether one uses Hunter’s “anticipatory set” or some other term, how the teacher 
addresses learning at the beginning of the class period is extremely important. Extend this thinking 
throughout the duration of a particular course and one would be well advised to model the 
appropriate behaviors early in the school year or semester. Opportunities to teach through 
modeling are greater at this time because rules and procedures are amenable to change (Doyle, 
1986). 

Setting the stage for learning when one is teaching on television becomes much more 
demanding of the teacher in terms of details that must be addressed. The flow of instruction must 
be planned in advance with materials and other strategies that do not require student interaction 
during the class period. The teacher must prepare visuals to illustrate processes that he/she might 
use to enhance learning. With the student role in learning largely dependent on the learner, the 
teacher will encounter difficult problems if the student has a motivation problem. This is true for 
any type of teaching situation, but teaching via distance learning formats tends to exaggerate the 
problem. Perhaps a more realistic approach for observing/planning and organization for 
instruction over a one-way video— two-way audio format would be to examine the materials 
themselves and try to determine the intent of the teacher. Observation could then discern if the 
teacher’s intent were effectively reached. While preparation for teaching without anticipating 
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student interaction may be necessary, the teacher would be well advised to plan for student 
interaction as much as possible. This may necessitate class assignments where the students are 
required to interact with each other via computers, or telephone or with materials the teacher has 
provided. Either way, the preparation for such occurrences will greatly enhance the chances for 
them to actually materialize. In some distance learning situations, teachers use “chat rooms” or 
“newsgroups” where the students interact via the computer between the times classes meet. When 
students log on, their messages are recorded and the teacher is able to make some assessment as 
to how much interaction is actually occurring. 

Effective teaching includes modeling on the part of the teacher. When one is observing 
instruction on television, an appropriate behavior might be to look for the teacher’s illustrations or 
demonstrations, or modeling behaviors as they are displayed for the students who sit in the studio 
where the televised teaching takes place. Interviews with T. Wiedmer & J.C Thompson Jr., 
university teachers who have taught on television, verify that students have the advantage of the 
verbal and nonverbal dimensions of the teaching act and are able to discern and communicate 
more clearly than those at distance sites (Personal Communication, Spring, 1997). Thus a 
criterion for observing teaching on television might be to determine how well the teacher utilizes 
the students who are present at the programming site for demonstrating teaching points he/she 
wants to make. How does one observe for “overlapping” behaviors on the part of the teacher on 
television? One suggestion might be to look for the teacher at the beginning of the class to start a 
review session by directing a question to the entire class; take part of the roll or pass out papers 
(perhaps through a fax machine) while the students are thinking about the answer (wait time); 
then call on a student whom the teacher thinks will answer. By utilizing strategies like this, the 
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review and take care of “housekeeping” chores at the same time. The stage is set for the students 
to think about their answers and compare them to the one given by the student called upon. In 
this way, unvoiced misunderstandings or incorrect answers can be corrected without visibility or 
embarrassment. On television this teaching situation will need to be modified to utilize the 
students who are in the studio to model the behaviors sought by the teacher. 

Modeling behaviors that teach desired outcomes might be observed as intentional or 
unintentional on the part of the teacher. How the teacher models respect for the students in the 
class sets the stage for how the students treat each other. In an attempt to explore the impact of 
invitational teaching practices on underachieving and apathetic teacher education students, Lange 
(1988) found a positive correlation between the invitational teaching techniques and increased 
positive experience for each student in the study. This study was an overt attempt to look at 
positive modeling behaviors and their impact on students who were disenchanted with their choice 
of a career. Students do not easily discern intentions unless the teacher exhibits overt behaviors. 
Overt teacher behaviors that are designed to enhance student motivation and attitude should be 
included in any evaluation paradigm of teaching on television. 

A research based teaching model indicates the need to check for understanding. In the 
one-way video— two-way audio format, opportunity to check for understanding might not be as 
applicable at the moment the teacher needs it as it would be in the regular classroom. The teacher 
therefore must make opportunities for this crucial step in the teaching process. Student 
interaction in distance ieaming situations is often accomplished via telephone or computer via 
telephone lines. If the students are to utilize the telephone to interact from distant sites, elapsed 
time may be dependent upon the number of lines accessible to the students. Further, responsive 
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telephone calls by students may consume so much time that the question or comment becomes out 
of place in the sequence of interactive events. Without the opportunity to check for 
understanding at the most opportune time, the teacher may not be as aware of the student’s 
comprehension of the content or processes being taught and may be unable to adjust the 
instruction for optimal learning. 

Programs which permit only selected on-line classes to call in can hardly pretend to be 
interactive when a majority of their students are simply watching a one-way TV instructional 
program that does not allow them to call-in ask questions or make comments. Another factor is 
whether or not students at a receiving site would be able to be on-line with students at other 
receiving sites at the same time, thereby enabling not only teacher/student interaction but also 
student/student interaction (Barker, 1988). 

Effective use of time is a variable necessary for successful teaching; however, when one 
observes teaching on television, there is the real iz ation that the use of time must be evaluated 
differently. When the teacher asks a question, three to five seconds of wait time is well 
documented in the literature (Rowe, 1996), but this is not realistic in a distance learning 
classroom. An adjusted expectation might well be as long as ten seconds or an adjusted definition 
of wait time might be developed to include student/teacher interaction via telephone. The obvious 
question arises, however, that if wait time must be extended, how does a teacher cover the 
necessary learning material needed in the course? Or, if the teacher adjusts and makes decisions 
to concentrate on more essential elements of the content, who participates in the decision as to 
what is essential? These are typical of the decisions teachers have to make (Hunter, 1982). 

Interviews with teachers who have taught on television indicate it is not possible to 
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process material initially, but teachers adjust over time and tend to acclimate to the demands of 
teaching on television. With practice and experience, improvement can take place. Strategies 
drawn from the research literature that might be applicable here include: 1) repeating content 
three or more times at spaced intervals results in content retention (Jersild, 1928), 2) Proactive 
markers such as “look at this,” “watch me,” or “now get this” increase retention also 
(Ehrensberger,1945), Petri (1963), and Maddox & Hoole, (1975). 

Teaching in a distance learning format that is one-way video-two way audio presents a 
different situation from that of the “normal” classroom. A study of communication always 
includes a discussion of how much of a spoken message is received and decoded through the 
nonverbal gestures or nuances that accompany it. Estimates range as high as 93% of a spoken 
message is received nonverbally. Teaching on television in a passive one-way video system 
mandates that the teacher must convey information or processes in a medium that restricts the 
nonverbal dimension of the communication process. In observing teaching, it is common to view 
a series of communication exchanges that result in teacher and student behaviors that can be 
codified. The observation may be accurate only to the extent that the observer has the opportunity 
to summarize the complete exchange. In observing teaching in a distance learning format, the 
observer will not have access to complete verbal and nonverbal exchanges and may have to 
extrapolate or hypothesize occurred. This raises the potential for differential and/or erroneous 
interpretation. 

The role of the student in distance learning is quite different from that of a live classroom. 
The student must cope with the technology to be able to participate in class assignments. 

Students must assume more responsibility for their own learning. Perhaps this is the greatest real 
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gain. Collaboration with the teacher to increase learning is essential in any learning situation. 
However, when the medium is a passive learning format like distance learning, more effort must 
be exerted by both the teacher and the learner. Precision of language used by the teacher when a 
concept is introduced and defined is important, but when examples and non-examples are given, it 
becomes critical. It should be noted that concepts are attained more completely when both 
examples and non-examples are provided (Tennyson, Woodley, and Merrill, 1972). 

Learning is enhanced when the teacher states each concept, law, or rule clearly. This might 
be accomplished by defining or stating the components, explaining in a language from which the 
students can profit or in such a way as to allow the students to demonstrate either by example or 
analogy. When students have opportunities for guided and independent practice, achievement is 
enhanced (Klausmeier, 1976). In 1979 Anderson, Evertson, and Brophy found a strong 
correlation between the number of minutes spent in guided practice (where a large number of 
questions were asked) and achievement. They also found a positive correlation between student 
achievement and the number of responses, suggesting interactive student practice at a brisk pace. 
Rosenshine and Stevens (1986) found that correctives and a high percentage of correct responses 
provided during guided practice are essential. The question then becomes: “How does the teacher 
give guided practice in a distance teaching/leaming format?” 

Independent practice used to extend content that has already been taught is a sound 
research based teaching practice. It usually involves two stages: 1) working on the first few 
examples or questions, and 2) when students have mastered the material and are working on 
reinforcement (Samuels, 1981). In the first stage, instruction is focused on the teacher’s behavior 
and is usually a direct instructional situation. An ineffective teaching behavior that is often 
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observed when students ask questions about material that has just been presented is that the 
teacher often repeats the first example that was used to explain the concept. Not only is the 
example repeated, it is often repeated more slowly, loudly or precisely. This practice is ineffective 
because the teacher is emphasizing an example that the student did not understand the first time it 
was presented. It is exacerbated by the fact that the teacher repeats it slowly and often more 
loudly as if to emphasize that the fault lies with the student and not the example. It is the wise and 
skilled teacher who recognizes this situation and shifts to another example while assuming the 
responsibility for the student not understanding. Frayer (1970) found that a few well-chosen 
examples were better than numerous examples and the number of examples that a teacher uses 
may not be as significant as the quality of examples given. 

In a distance learning format, the teacher will be hard pressed to incorporate independent 
practice while the lesson is being taught. Learning then entails work or practice beyond the 
classroom. Teaching on television can be greatly enhanced by use of computer-assisted 
interaction. Riel (1993) found computer-mediated communication raised cross-cultural 
awareness. Social interaction is increased (Johnson-Lenz & Johnson-Lenz, 1993), writing skills 
are enhanced (Paulsen, 1992), knowledge construction and thinking are facilitated, and 
independent learning strategies are developed (Mason & Kaye, 1989). However, lack of visual 
cues can also inhibit depth of communication (Selfe & Meyer, 1991). From these studies an 
evaluation of teaching on television, especially when the evaluator is trying to assess independent 
practice, clearly must attempt to assess activities that occur beyond the classroom and between 
times when the class meets. Interaction between students outside the classroom may be a direct 
reflection of the teacher’s action during classtime. 
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Instruction via distance learning is a widespread practice among educational institutions. 

It is utilized for upper level classes at the high school level to provide opportunities for students 
who plan to pursue higher levels of study. Colleges and universities use distance learning for a 
variety of instructional purposes from delivering classes within state borders and between states 
nationwide to video conferencing that might have international hookups. The explosion of 
technology has served to remove the barrier of distance for access to learning. The speed with 
which electronic signals can transmit information has permeated nearly all areas of society; 
education is no exception. Technology is here to stay and its implications for instruction stagger 
the imagination. 

An issue of distance learning that must be considered is the quality of the instruction and 
learning that takes place. The research on evaluating instruction via distance learning formats has 
been largely confined to analyses of self reported data from participants. Careful observation with 
well researched criteria are needed to evaluate the effectiveness of distance learning instruction. 
The research on effective teaching is a body of knowledge that has been accumulated largely in 
traditional classrooms. The distance learning site is not a traditional classroom as far as teaching 
and learning are concerned, and the traditional criteria utilized for observing and evaluating 
teaching are not appropriate for observing and evaluating instruction via distance learning. As 
evaluators use the results of research on effective teaching and apply them to distance learning 
instruction, modifications of existing criteria need to be made. There is also a need for new 
criteria that are more compatible with distance learning environments. 

Student attitudes toward learning are especially important for success in a distance 
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learning situation. The teacher and learner must possess a degree of confidence and comfort with 



the technology. When the nonverbal dimension of teaching is missing, the preparation for 
instruction is different and must focus more on created visual images rather than direct 



observation. 
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